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ABSTRACT 

Middleboxes are special network devices that perform vari¬ 
ous functions such as enabling security and efficiency. SDN- 
based routing approaches in networks with middleboxes need 
to address resource constraints, such as memory in the switches 
and processing power of middleboxes, and traversal con¬ 
straint where a fiow must visit the required middleboxes in 
a specific order. In this work we propose a solution based 
on MultiPoint-To-Point Trees (MPTPT) for routing traffic in 
SDN-enabled networks with consolidated middleboxes. We 
show both theoretically and via simulations that our solu¬ 
tion significantly reduces the number of routing rules in the 
switches, while guaranteeing optimum throughput and meet¬ 
ing processing requirements. Additionally, the underlying 
algorithm has low complexity making it suitable in dynamic 
network environment. 
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1. INTRODUCTION 

Middleboxes (e.g. proxies, firewalls, IDS, WAN op¬ 
timizers, etc.) are special network devices that per¬ 
form functional processing of network traffic in order 
to achieve a certain level of security and performance. 
Each network flow may require certain set of functions. 

In some cases these functions can be applied only in a 
particular order, which makes routing in networks with 
middleboxes under limited resources constraints even 
a more difficult task. Mechanism of controlling rout¬ 
ing through the specified functional sequence is called 
Service Eunction Chaining (SEC). Logically centralized 
traffic control offered by SDN enables traffic routing op¬ 
timization (in terms of device costs, total throughput, 
load balancing, link utilizations, etc.), while satisfying a 
correct traversal of network middleboxes for each flow. 
Several recent works (e.g. pT] , provide relevant 

solutions. 

Eunctionality provided by middleboxes can be incor¬ 
porated in the network in several ways. Traditional 
middlebox is a standalone physical device that can typi¬ 


cally perform one network function, and may be located 
at an ingress switch. With the development of the Net¬ 
work Eunction Virtualization (NEV), middleboxes may 
be implemented using Virtual Machines (VMs) that can 
be flexibly installed at the Physical Machines (PMs). In 
addition, virtualization enables implementation of the 
consolidated middleboxes 
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where a flow receives all 
of its required service functions at a single machine. The 
consolidated middlebox model simplifies traffic routing 
and helps reduce the number of routing rules in the 
switches. 

In this paper, we follow the model in [^, and as¬ 
sume that each middlebox function is an application 
that can be installed at certain VMs within the PMs. 
It is also assumed that every flow obtains all its re¬ 
quired functional treatment at a single PM, and thus 
the consolidated middlebox model is implied in the pa¬ 
per. Network function consolidation and flexible imple¬ 
mentation of middleboxes were previously discussed, for 
example in HI, H, i and 


Depending on the network traffic environment, two 
types of routing schemes can be developed: offline, where 
all required traffic demands are given or can be esti¬ 
mated (for example, using a service level agreement be¬ 
tween the customer and the provider), and online, where 
demands are unknown and a routing solution for each 
coming flow is made based on the flow class and the 
current state of the network. A solution obtained by a 
routing scheme can be converted into a set of routing 
rules that are installed in the switches. Different criteria 
can be used to characterize the achievable performance 
of a routing scheme: total throughput, average delay, 
maximum PM utilization, etc. Besides achieving a de¬ 
sired network performance, a routing scheme must also 
satisfy resource and routing constraints. Additionally, 
three new constraints are of a special interest in the 
SDN-enabled networks with middleboxes. 


• Sw'itch memory capacities: number of rules in¬ 
stalled in a single switch is limited by its memory 
capacity. Ternary Content-Addressable Memory 
(TCAM) used in SDN switches is a scarce resource 
which is expensive both in terms of cost and power 




m{vj) 

capacity of switch Vj G Vsw 

r(vj) 

number of rules in switch Vj G Vsw 

Kvj) 

capacity of PM Vj G Vpm 

9{e) 

capacity of link e 

corrii 

^ Si-iti-) di, Ci ^ 

commodity i 

with source Si, destination ti, 
demand di, and class q 

Pi 

cost (in PM resources) of corrii 

M 

total number of commodities 

c 

number of different traffic classes 

Vt 

set of distinct destinations 


Table 1: Main notations. 


consumption. 

• Middlebox processing capacities: load on each 
middlebox should not exceed its processing capac¬ 
ity. Overload of middleboxes has to be avoided 
since it may cause loss of traffic, delay, incorrect 
traversal sequence or other problems. 

• Traversal constraints: required network func¬ 
tions have to be applied to any given flow in a 
correct order. 

The switch memory constraint is important: flow ta¬ 
ble overflow is a serious problem that can significantly 
degrade network performance and, therefore should be 
avoided. Because this constraint is of integer type, it 
makes the problem of finding an optimal routing solu¬ 
tion hard. If, in addition, middleboxes are added to the 
network, finding such routing becomes even harder. 

In this paper we present an approach based on multi- 
point-to-point trees that efficiently finds a routing with 
a guarantee on the maximum number of rules in a single 
switch, while satisfying all other network constraints. 
Moreover, our routing solution scales well with the net¬ 
work size: the explicit bound C + 2|^o| + |^t| — 
on the number of rules is additive and depends linearly 
on the number of destination nodes (|1^|), links (|^o|) 
and flow classes (C) in the network. 

This paper is organized as follows: in Section 2 we 
introduce our network model and necessary notations. 
In Section 3 we describe our routing solution, in Sec¬ 
tion 4 we evaluate its performance by simulations and 
demonstrate its advantages over several other routing 
schemes. Finally, we compare our solution with related 
works in Section 5, and conclude in Section 6. 

2. PROBLEM FORMULATION 
2.1 Network Topology and Resources 

We assume that the network topology is defined by a 
directed graph Go = (Vb, Eq), where Vb is the set of its 
nodes and Eq is the set of edges. Each node corresponds 


either to a switch or to a PM, and each edge is a link 
connecting either two switches, a switch with a PM, or a 
PM with a switch. We denote by Vsw and Vpm the node 
sets corresponding to switches and PMs, respectively, so 
that Vsw U Vpm = Vo, and Vsw H V^m = 0- If will be 
assumed for simplicity that each PM is connected with 
a single switch by bi-directional links as shown in Fig. 
[T^ Let Vsw^pm be the subset of nodes in Vsw fbat 
are directly connected to the PM nodes {Vsw^pm = 
{swl^ sw2^ swV} in Fig. la). 

Each switch has a certain memory capacity that can 
be expressed as a number of rules that it can accom¬ 
modate. We will denote this number by m{vj) for a 
switch located at node Vj {j = 1,..., \ Vsw\)^ where \A\ 
is the cardinality of a set A. Additionally, let r{vj) be 
the number of rules in this switch in a routing solution. 

Although a PM may have several types of resources 
(e.g. memory, CPU), it will be assumed for simplicity 
that each PM is characterized by a single resource ca¬ 
pacity that will be denoted by b{vj) for a PM located at 
node Vj (j = 1,..., |Vbm|)- Similarly, each link G Eq 
{ k = 1,..., |Fo|) has an associated link capacity that 
will be denoted by g{ek). 


2.2 Network Functions and Commodities 

There exist several types of network functions (fire¬ 
wall, IPS, IDS, WAN optimization, etc.), and each func¬ 
tion has its own cost per unit of traffic in terms of PM 
resources. Although in this work we assume that this 
processing cost is the same for all PMs, it is easy to 
generalize it to the case when the costs are distinct for 
different PMs. 

Additionally, there is a set of M traffic demands or 
“commodities” that have to be routed in the network. 
We will use the terms traffic demand and commodity 
interchangeably. Commodity corrii is defined by a four¬ 
tuple corrii = < Si, ti, di, Ci >, where i = 1,..., M. Here 
Si G Vsw and ti G Vsw are, respectively, source and des¬ 
tination nodes, di is an amount of flow that has to be 
routed for commodity corrii, which we will call commod¬ 
ity’s demand, and q is an ordered set of network func¬ 
tions required by this commodity. Any such ordered 
set of network functions defines the class of a commod¬ 
ity. We will denote by C the total number of different 
classes of traffic demands. Due to various functional re¬ 
quirements, different commodities may have different 
per unit of traffic costs in terms of PM’s processing 
power. Let p{i) be such cost per unit of traffic for traffic 
demand corrii. 

Each PM hosts at most C VMs, where a single VM 
corresponds to a single commodity class. It is assumed 
that when a packet from a commodity of class k arrives 
to a PM, it is transfered to the virtual machine asso¬ 
ciated with class k, and all network functions of class 
k are applied to this packet in a correct order. Distri- 













bution of each PM’s processing capacity among C VMs 
has to be determined. It is assumed, however, that po¬ 
sitions of PMs (nodes Vpm) are given as an input and 
are not subject to change. 

By Vt we will denote the set of distinct destinations, 
then |Vt| < M, |Vt| < |Kw|- Main notations are sum¬ 
marized in Tabled] 

2.3 Routing via Integer Linear Optimization 

In this work we employ the idea of consolidated mid- 
dleboxes, and each packet belonging to corrii gets all 
functional treatment specified by q at a single PM. It 
is allowed, however, that a single commodity’s traffic is 
split into several paths from Si to and distinct paths 
may traverse distinct PMs. We point out that splitting 
occurs at the IP flow level and not at the packet level. 
This is similar to Equal Cost Multipath in Data Cen¬ 
ters, where hashing is used to split traffic at the IP flow 
level for routing on multiple paths. 

If the traffic demands are known in advance, an opti¬ 
mization problem can be posed whose feasible solution 
defines a routing that satisfies all network constraints. 
The variables of this optimization problem /f (e) are the 
amount of traffic of commodity corui on edge e G Eq, 
i = 1,..., M. Here superscript x G {0,1} with zero 
value corresponds to the traffic that has not visited a 
consolidated middlebox, and unit value is used to de¬ 
note the traffic that has been processed by the required 
network functions. There are thus 2 • M • \Eq\ variables 
in this optimization problem. Let di{v) be the demand 
from a node v G Vsw foi* the commodity corrii. Note 
that di{v) = di if V = Si^ and is zero, otherwise. The 
problem is formulated as follows. 

ILP Optimization (1): 

min 

eeEo,l<i<M, 

xe{0,l} 

\/v G Vsw, Vi : 1 < i < M : 

E E (la) 

(v,w)eEo (u,v)eEQ 

\/v G Vsw • V Mi : 1 < i < M : 

E Yi fhu,v)=0, (lb) 

{v,w)eEo {u,v)eEo 

Vf e Vpm, Vi : 1 < i < M ; fl{u,v) = 0, (Ic) 

Vi; G Vpm, Vi : 1 < i < M : f°{u,v) = fl{v,u), (Id) 

Vi; G Vpm ■ Y - ^(^)’ (1®) 

l<i<M 

\/eGEo: Y /°(e)+ E < die), (H) 

l<i<M l<i<M 

Vv G Vsw ■ r(v) < m(v), (Ig) 

Ve G -Bo, Vi : 1 < i < M, Vx e {0,1} : /f (e) > 0. 

(Ih) 


pml 


pm2 



graph Go = (Vo,Eo). 


at Step 1 . 



(c) Graph G2 = (V2,E2) 
constructed from graph 
Gi at Step 2 . 


Figure 1: Example of a given graph Go and graphs Gi 
and G 2 constructed at the first and the second steps of 
our algorithm, respectively. 


Constraints (la) and (lb) are flow conservation con¬ 
straints for switches, constraint (Ic) forbids the traffic 
that has already been processed by a middlebox (PM), 
to visit a middlebox again. Next constraint ( |ld| ) says 
that all unprocessed traffic becomes processed at the 
PM associated with node v G Vpm- Further, constraint 
(le) is a PM processing capacity constraint. The fol¬ 
lowing constraint 0 is a link capacity constraint, and 
condition (Ig) corresponds to the switch memory con¬ 
straint. Finally, requires that all flow values are 
nonnegative. The objective function of this optimiza¬ 
tion problem is the total flow over all edges. This choice 
of the objective function guarantees that no cycles will 
exist in an optimal solution. Notice that there is no con¬ 
straint f^(v, u) = 0 similar to constraint (Ic), because it 
will be automatically satisfied due to the optimization’s 
objective function. 

Solution to this optimization problem expressed in 
terms of variables /f (e) can be translated to a path- 
flow formulation [^, and the routing rules in switches 
can be obtained that implement this path-flow solu¬ 
tion. Each routing rule in a switch corresponds to a 
single path in the path-flow solution. Notice that in the 
solution to the optimization problem, more than one 
source-destination path can be used to transfer traffic 
for a single commodity. 

The optimization problem formulated above contains 
integer switch memory constraints (Ig) and thus be¬ 
longs to the class of Integer Linear Programs (ILP). 
This problem, therefore, is NP-hard, and it is extremely 






















difficult to obtain its solution. In this work, we adapt 
the idea of Multipoint-To-Point Trees to construct a 
feasible routing scheme for SDN-enabled networks with 
middleboxes and known traffic demands. Although the 
integer switch memory constraints are not explicitly in¬ 
corporated into our solution, we can obtain the worst 
case bound on the number of rules in each switch. More¬ 
over, we show that this bound scales well with the net¬ 
work size and is low enough for our routing scheme to 
be implemented in the networks with existing switches. 

3. SOLUTION OVERVIEW 



Figure 2: Schematic of the MPTPT-based routing al¬ 
gorithm. 


3.1 MPTPT Approach 

In this work we take advantage of the capabilities pro¬ 
vided by SDN to design efficient routing. In particular, 
SDN facilitates global design optimization based on in¬ 
puts and measurements collected from various points of 
the network, and the ability to translate design solutions 
into rules which can be downloaded to the switches. 
One of the major components of our routing solution 
is multipoint-to-point trees that were previously used, 
for example, by the label based forwarding mechanism 
of MPLS prl. Each multipoint-to-point tree is rooted 


at some node, and all its edges are oriented towards 
this root node. Such trees can be used to route traffic 
from several sources to a single destination, and each 
tree is assigned with its own tag which is used to label 
all traffic belonging to this tree. Utilization of MPTPTs 
helps to reduce the number of routing rules in the whole 
network [^. 

Our solution contains two main steps. These steps are 
purely computational (not actual routing steps), and 
allow to determine how the traffic for each commodity is 
labeled and routed. At the first step we route all traffic 
from the sources (i = 1,..., M) to PMs. At the 
second step, we route all traffic that has been processed 
by the required network functions during the first step 
from the PMs to the corresponding destinations U, (i = 
1,..., M). Both steps involve construction of MPTP 
trees: there are C roots for multipoint-to-point trees 
built at the first step, where each root corresponds to 
a particular flow class, and there are \Vt\ roots for the 
trees at the second step. There can be in general more 
than one MPTP tree rooted at a single node. In Fig. 
[^we show the schematic of our MPTPT-based routing 
algorithm. 


3.2 Step 1: Routing from Sources to PMs 

At the first step we consider a graph Gi = (Ui,Ei) 
which is obtained from the initial graph Go as follows: 
we add G additional nodes ,..., Vc such that node 
Vk corresponds to the traffic class k. This set of G new 
nodes is denoted by Vp, and \Vp\ = G. We further 
remove ”PM” nodes belonging to the set Vpm, together 


with the edges going to and from these nodes. Then, 
we connect each node from Vsw^pm by edges to every 
node from Vp. These new edges are not assigned with 
capacities explicitly, but the maximum amount of flow 
on them will be determined by the capacities of PMs 
and the capacities of removed links from graph Go that 
were connecting nodes in Vsw^pm with nodes The 
vertex set of graph Gi is a union of node sets Vsw and 
Vp: Vi = Vsw U Vp. Number of links in the graph Gi 
is \Ei\ = \Eo\ + \ Vpm\ • (G — 2). In Fig. [^we show an 
example of a network topology defined by a graph Go 
(Fig. la) and corresponding constructed graph Gi (Fig. 


lb). In this example it is assumed that there are two 


classes of flows and the nodes Vi and V 2 are associated 
with flow classes one and two, respectively. In Fig. 
the new added links are shown by dashed arrows. 

We additionally modify destinations of the given com¬ 
modities. In particular, destination of all traffic de¬ 
mands of class k is node Vk, k = 1,..., G. Therefore, 
for each commodity com^, its destination is one of the 
nodes in Vp. We can now formulate an LP optimization 
problem that we solve at the first step of our method. In 
contrast to the commodity-based ILP problem consid¬ 
ered in the previous subsection, the optimization here is 
in a tree-based formulation, and we do not distinguish 
traffic from different sources if they are for the same des¬ 
tination, i.e. if they belong to the same network class. 
Let V denote a PM connected to node v £ Vs 


sw^pm 


m 


graph Go (for example, v = pm3 for v = sw6 in the 
example from Fig. [^, and p(t), where t e Vp^ denotes 
the cost of PM resources per unit of traffic of class cor¬ 
responding to the node t. 


LP Optimization (2) of Step 1: 

min ft{e), 

eeEi,teVr> 

yt e Vp, yveVi, v^t: 

Y Mv,w)- Y ft(u,v) = dt(v), (2a) 

{v,w)^Ei {u,v)^E\ 

Ve e -El n : Y 

tev-p 













Vi; G Vsw^pm : ^ < mm{g{v,v),g{v,v)}, 

tev-p 

(2c) 

Vv G Vsw^pm ■■ ^ p{t) ■ Mv, t) < b{v), (2d) 

teVv 

Me G El, Mt eVp : ft{e) > 0. (2e) 


In this optimization problem variable /t(e) is an amo¬ 
unt of flow to destination t G Vp on link e G Ei. Con¬ 
straint (2a) is a flow conservation at node v, condition 
(2b) is a link capacity constraint that should be sat¬ 
isfied for any link that belongs to the both edge sets 
Eq and El of graphs Go and Gi, respectively. Further, 
constraint (2c) is a link capacity constraint for the links 
that connect switches with PMs in graph Go- This con¬ 
straint is necessary for feasibility of the solution to op¬ 
timization problem (2) in the original graph Go- Notice 
that in the right hand side of (2c) there is a minimum 
between capacities of the links going from a switch to 
a PM and from a PM to a switch. It will guarantee 
that all traffic processed at a PM can be send back to 
a switch connected to this PM. Next constraint (2d) is 
a PM capacity constraint, and by (2e) we require that 
flow on each link is nonnegative. As in the ILP opti¬ 
mization problem (1), we minimize the total network 
flow to avoid cycles. 

Solution to the optimization problem (2) determines 
how the traffic is routed from the sources to the PMs. 
Using Algorithm Flow2Trees(t) from that is listed 
as Algorithm below for completeness, from a basic 
feasible solution to the LP (2) we construct 

multipoint-to-point trees rooted at the destination nodes 
from Vp, so that all network traffic in the solution is dis¬ 
tributed among these trees. Each tree contains traffic 
of the same class, leafs of a tree are the sources for this 
traffic class, and amount of traffic from each source in 
any tree can be determined. It is possible that several 
Vsw^pm nodes belong to the same tree, i.e. one tree can 
route traffic to several PMs. Algorithm is iteratively 
applied to construct trees to each destination t £ Vp. 
We will provide an upper bound on a total number of 
trees in the subsection 3.5. We refer the reader to 
for the details and analysis of Algorithm 


Algorithm 1: Flow2Trees(t) 

Input :G = {V,E),t, ft{e) {MeeE). 

Output: Set of MPTP trees rooted at t and 
containing all traffic to t. 

1 while there is a source s with demand to t do 

2 using only edges e with flow to t > 0), 

construct a tree R to t spanning all sources 
with demand to t; 

3 move as much flow as possible to R; 

4 end 


tl t2 



(a) Network topol¬ 
ogy, Go = {Vo,Eo). 


Vi 



s 


(b) Graph Gi = 
(Vi^Ei) and trees 
Ri, R 2 obtained at 
Step 1. 


Figure 3: Example that shows possible ambiguity in 
commodity assignment at Step 2. 


3.3 Step 2: Routing from PMs to Destinations 

At the second step of our algorithm we use MPTP 
trees to route traffic from the PMs to destinations in 
graph G 2 obtained from Gi as follows. Eirst, nodes Vp 
and links to them are removed from the network. There¬ 
fore, the node set of the resulting graph G 2 = (V 2 ,E 2 ) 
only contains nodes from Vsw- V 2 = Vsw Number of 
links in graph G 2 is |F; 2 | = \Ei H Eo\ = \Eo\ - 2 • |Up^|. 
Second, the link capacities are updated: for each link 
e, the amount of traffic on it in the solution to (2) is 
subtracted from this link’s initial capacity g{e). We will 
denote by g{e) the updated capacity of link e. Graph 
G 2 corresponding to graph Go from Eig. [^is shown in 
Fig. 

We then create a set of commodities for the sec¬ 
ond step. It is assumed that all traffic processed at 
a PM V returns to switch v G Vsw^pm connected to it. 
Therefore, all traffic at Step 2 is routed from the nodes 
ysw^pm to the destinations U, where i = 1,..., |Vt|- 
Solution to optimization (2) determines amount of traf¬ 
fic of every class and from every source arriving to each 
PM. However, amount of traffic to each destination ti 
arriving to a PM, in some cases can not be determined 
unambiguously. This can happen when there exist more 
than one commodities with the same source and of the 
same class but with different destinations. We illustrate 
this possibility with an example from Eig.[^ In Eig. [3a| 
the network topology is shown: there is only one source 
node s, two PMs and two destination nodes tl and t2. 
It is also assumed that there is only one traffic class 1, 
30 units of traffic from s should be sent to tl, and 70 
units to t2. Graph Gi constructed at the first step of 
our algorithm is shown in Eig. [31^ Suppose that two 
trees to node Vi were obtained at the first step: tree R1 
[s ^ vl ^ Vi), and tree R2 {s ^ v2 ^Vi). The links 
belonging to the trees R1 and R2 are shown by solid 
green (tree Rl) and dashed blue (tree R2) lines in Eig. 
[ 31 ^ Assume for example, that 40 units of traffic of class 












1 belong to the tree and 60 units belong to the tree 
R2. Therefore, after Step 1 of our algorithm it is known 
how much traffic of this class from source s arrives to 
node vl (to be processed at PMl), and how much traf¬ 
fic arrives to node v2 (to obtain functional treatment 
at PM2), but distribution of traffic by destination at 
nodes vl and v2 is unknown. This information, how¬ 
ever, is necessary to define commodities at the second 
step of our approach, and thus a distribution decision 
is required. 

We will use the following heuristic to determine the 
traffic distribution by destination at each node v G 
Let i? be a set of trees obtained at Step 1 
of our algorithm that carry traffic of the same class c 
from a source node s to the root node Vc correspond¬ 
ing to this traffic class. In addition, let T be the set of 
destinations of commodities with source s and of class 
c, and di,..., d\T\ are corresponding demands. By the 
definition of a tree, in each tree Ri from set i?, there is a 
unique path from s to Vc^ and therefore, all traffic from 
s in the same tree obtains functional treatment at a 
single PM. According to our heuristic, in each tree 
amount of traffic to destination t G T is proportional 
to the fraction of traffic to this destination in the total 
amount of traffic to all destinations, i.e. proportional 

Ti n 


to dt/ ^ di. In example from Fig. 
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R — 


i=l 


Vc = Vi^ T = di = 30 and^d 2 = 70. Then, 

according to the heuristic, in tree Ri\ 30/100 • 40 = 12 
units of traffic are to destination ti, 70/100 • 40 = 28 
units of traffic are to destination ^ 2 - Similarly, in tree R 2 
the distribution is 30/100 • 60 = 18 and 70/100 • 60 = 42 
units to ti and ^ 2 , respectively. 

Using this distribution heuristic, we form a set of 
commodities for the second step of our algorithm. At 
the Step 2 we do not distinguish traffic from different 
sources and from different network classes if they have 
the same destination. We construct MPTP trees with 
the roots at the destinations U, i = 1,..., |Vt|- Simi¬ 
larly to Step 1, we first solve the following LP: 


LP Optimization (3) of Step 2: 

min E /<(')' 

66-^2 ,tG Vt 

Vi e Vt, 'iveV2,vi=t-. 
y] ft{v,w)~ ft{u,v) = dtiv), 

{v,w)eE2 {u,v)eE2 

Vee^2: 

tEVT 

Ve G -E’ 2 , Vt G Vt • ^ 0- 


(3a) 

(3b) 
(3c) 

Here (3a) and ( |3b| ) are flow conservation and link ca- 
pacity constraints, respectively, and (3c) is a require¬ 
ment for flows to be non negative on each link. Using a 
basic feasible solution to this problem, we apply again 
Algorithm and obtain another set of multipoint-to- 


point trees. Complete version of our MPTPT-based 
routing approach is summarized in Algorithm 


Algorithm 2: MPTPT-Based Routing 
Input : Go = {Vq^Eq)^ commodities corrii 
(t = l,...,M). 

Output: Set of MPTP trees rooted at PM nodes 
and destination nodes. 

1 Step 1: routing from sources to PMs: 

2 construct graph Gi = (Vi,F^i) from 
Go = (Uo,L^o); 

3 obtain commodities for Step 1; 

4 find a basic feasible solution to LP (2); 

5 find MPTP trees for the solution to LP (2) using 
Algorithm 

6 Step 2: routing from PMs to destinations: 

7 construct graph G 2 = {V 2 ^E 2 ) from 
Gi = (Ui,F;i); 

8 obtain commodities for Step 2; 

9 find a basic feasible solution to LP (3); 

10 find MPTP trees for the solution to LP (3) using 
Algorithm 


After both steps of our algorithm are performed, we 
can determine for any initial commodity < q > 

what trees carry its traffic to the destination Each 
commodity’s packet is assigned with two tags at the 
source switch: one for a tree label from Step 1, and an¬ 
other one for a tree label from Step 2. The first label 
can be removed from a packet during functional pro¬ 
cessing at a PM, and therefore the maximum number 
of routing rules in a single switch does not exceed the 
total number of multipoint-to-point trees of both steps. 
As suggested in previous works (e.g. [^), VLAN and 
ToS fields of a packet header can be used for labels. 

3.4 Analysis 

In this subsection we provide and prove an upper 
bound on the total number of MPTP trees generated 
by Algorithm Each tree has its own label and any 
switch may contain at most one routing rule correspond¬ 
ing to this tree. The bound, therefore, also limits the 
number of routing rules in any switch. 

Proposition 1. Number of MPTP trees produeed by 
Algorithm^ does not exeeed G + 2|Eo| + |Vt| — 2|l^rn|- 

Proof. It was shown in that when Algorithm 
is iteratively applied to a basic feasible solution of the 
multicommodity flow problem (3), the maximum possi¬ 
ble number of created trees is \Vt\ + \E 2 \^ i-e. bounded 
above by the sum of number of destinations and num¬ 
ber of links in a network. The second term in this sum 
(IE 2 I) corresponds to the number of bundle constraints 
in LP. A constraint is called bundle if it involves vari¬ 
ables for different destinations. In optimization problem 









(3) link capacity constraints ( |3b| ) are bundle, and there 
are |^ 2 | such constraints. Although optimization prob¬ 
lem (2) is slightly different from (3), a similar bound 
for it can also be established. Number of bundle con¬ 
straints in (2) is |Eo| - 2 • \Vpm\ + \Vpm\ + \Vpm\ = \Eo\, 
and number of destinations is equal to the number of 
traffic classes C. Therefore, the total number of trees 
produced by Algorithm is C + |^o| + |Vt| + |^ 2 | = 
C + 2|^o| + |^T|-2|y^^|. □ 

Notice that while our bound depends on the number of 
classes C, it does not depend on the number of com¬ 
modities, because |Vt| is bounded by The bound 

is additive and thus scales well with the network size. 
Moreover, as shown by simulations, the real number 
of routing rules obtained by our algorithm is generally 
much smaller than this worst case bound. It is crucial 
that a basic feasible solution is used as an input to the 
Algorithm^ at both steps of AlgorithmWe refer the 
reader to ^ and for a more detailed discussion of 
basic feasible solutions and bundle constraints. 

Therefore, Algorithm 2 efficiently solves a routing 
problem (it contains two linear optimizations and Algo¬ 
rithm 1 with polynomial time complexity) with a guar¬ 
antee that the number of routing rules in each switch is 
limited by an additive bound. 

4. EVALUATION 

In this section we evaluate the performance of Algo- 
rithm|^and compare it with three other routing schemes. 
The first routing scheme is defined by optimization prob¬ 
lem (1) with relaxed integer switch memory constraint, 
and a basic feasible solution for it is found using simplex 
method. The second scheme uses the same relaxed LP, 
but an interior point method (IPM) is applied to find a 
solution. Finally, the third scheme is based on a greedy 
shortest path approach. In this approach the commodi¬ 
ties are initially sorted in descending order by their to¬ 
tal PM capacity requirement. Then, iteratively for each 
commodity a shortest path is found from its source to a 
PM, and then a shortest path from the PM to commod¬ 
ity’s destination. If link and PM capacity constraints 
on the shortest path do not allow to send commodity’s 
total demand, a maximum possible fraction of it is sent 
along this path, and the remaining traffic is sent along 
the next shortest paths until all commodity’s demand 
is routed. If at some point there is no path available 
to send commodity’s residual demand, the algorithm 
stops. 

Our evaluation analysis consists of two experiments. 
In the first experiment we find routing solution using 
each of four algorithms and calculate an average number 
of routing rules in switches for each solution. Second ex¬ 
periment allows to estimate for each routing algorithm 
the maximum total throughput that it can route. Both 


experiments are carried out for three network topolo¬ 
gies: example from Fig. Geant topology, and fat tree 
topology. Geant network contains 41 switch nodes and 
9 additional PM nodes that are connected to 9 switch 
nodes having the highest nodal degree (so that each PM 
is connected to exactly one switch). Fat tree topology 
consists of 22 switch nodes (2 core, 4 aggregation and 
16 edge switches), and 6 PM nodes such that each PM 
node is connected to a single core or aggregate switch 
node. Link and PM capacities were fixed in each sim¬ 
ulation, and took values, respectively, 100 and 500 for 
the network on Fig. and 500, 500 for Geant topol¬ 
ogy. For the fat tree topology links between core and 
aggregation switches had capacities 200, links between 
aggregation and edge switches had capacities 10, and 
links between switches and PMs were fixed at 100. In 
addition, each PM had capacity 500. 

Experiment 1: Average Number Of Routing 
Rules. In the first experiment we varied number of 
classes and number of commodities, and each commod¬ 
ity’s source, destination and class were generated ran¬ 
domly. The demands of the commodities, however, were 
all equal and fixed at 0.2. Results of Experiment 1 for 
3 and 7 traffic classes are shown in Fig. It can be 
observed from the results that Algorithm allows to 
reduce average number of routing rules in switches by 
a factor of up to 10. We did not add plots correspond¬ 
ing to the interior point method solution for Geant and 
fat tree topologies because in the IPM solution average 
number of rules is much higher compared to the other 
algorithms. We also performed simulations for one and 
five traffic classes, and the results look similar to Fig. 
m The values of bounds on the maximum number of 
rules in switches are 43, 295 and 137 for the topolo¬ 
gies in the same order they are presented in Fig. [^and 
for 7 traffic classes. These values were obtained under 
assumption that \Vt\ = |Esw| and therefore, limit the 
number of routing rules in each switch for any arbitrary 
large number of commodities. 

Experiment 2: Maximum Total Throughput. 
In the second experiment we measured the maximum 
total throughput that can be routed in a network by the 
Algorithm]^ Notice, that ILP (1) with relaxed switch 
memory constraint always finds a routing solution when 
it exists. Therefore, we used the relaxed LP (1) to deter¬ 
mine the maximum possible network throughput. For 
a given set of commodities, we increased iteratively de¬ 
mands of all commodities by the same value until the 
relaxed LP (1) became unfeasible. We stored this maxi¬ 
mum demand value, and repeated the procedure for the 
Algorithm and also for the Greedy Shortest Path al¬ 
gorithm. Results provided in Fig. [^demonstrate that 
a loss in maximum throughput of the Algorithm is 
relatively small. 








(a) Network topology from Fig. 


(b) Geant topology. 


(c) Fat tree topology. 


Figure 4: Comparison of average numbers of routing rules in switches for three and seven traffic classes. 


5. RELATED WORK 

Most of the previous works on routing in networks 
with middleboxes aim to achieve a fair load balance 
among middleboxes [^, [^. However, it is generally 
assumed that for each given commodity a set of proper 
paths is provided (or a single path) [^, [^, [^. Al¬ 
though this assumption simplifies finding a routing, it 
has an important disadvantage: it is generally not easy 
to find a set of suitable paths for all commodities such 
that all traffic demands can be routed and network con¬ 
straints are satisfied. In the problem setup is similar 
to ours: middleboxes run as virtual machines at the 
PMs, traffic demands are known, and a routing linear 
optimization program is proposed. The integer switch 
memory constraint, however, is not incorporated into 
the routing problem, what makes it easier to find a fea¬ 
sible solution. The authors also explore the problem of 
an optimal placement of middleboxes. 

An optimization model for Network Intrusion De¬ 
tection Systems (NIDS) load balancing is presented in 
[^. A linear optimization problem contains estimates 
of the commodities’ demands and thus is designed for 
a carrier-grade traffic environment. It is solved periodi¬ 
cally to remain an optimal feasible routing for changing 
traffic demands. This optimization problem does not 
include switch capacity constraints, and its goal is to 
balance the load among different NIDS. It is also as¬ 
sumed that for each commodity a precomputed path is 
given as an input. Therefore, the problem solved in 
is not exactly a traffic engineering problem, but a load 


engineering. 

Switch memory constraint was taken into account in 
10 , and it was also assumed that there exists a set of 


suitable paths for each commodity. Due to the difficulty 
of the original optimization problem, it was decomposed 
in 10 into two stages. At the first (offline) stage, only 


the switch memory constraint is taken into account, and 
for each commodity a subset of paths is chosen from its 
original path set. Since the switch memory constraint 
is integer, the whole offline optimization problem solved 
at the first stage is an Integer Linear Program (ILP). 
At the second (online) stage, a simpler Linear Program 
(LP) is formulated to solve a load balancing problem. 


6. CONCLUSION 

In this work we proposed a multipoint-to-point tree 
based algorithm for SDN-enabled networks with mid¬ 
dleboxes and given required traffic demands. We showed 
both theoretically and experimentally that in the rout¬ 
ing solution obtained by our algorithm, the maximum 
number of routing rules in a single switch is bounded, 
and this explicit bound scales well with the network 
size. Moreover, the low complexity of the algorithm al¬ 
lows its application the algorithm in dynamic network 
environment. 
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